IoUring: use typed Flags and start syncing with liburing 2.12 #25394

bernardassan · 2025-09-28T23:11:09Z

IoUring Overhaul

Future Enhancement

Indicate what versions of the kernel support the new flags and operations
Port all test and examples from liburing 2.12 to IoUring
Implement functionalities that require int_flags and enter_ring_fd
The rest ...

daurnimator · 2025-09-29T06:53:35Z

Move io_uring_sqe.zig into IoUring

That seems like a regression? Why is that desirable?

bernardassan · 2025-09-29T09:25:24Z

Why is it a regression? It makes IoUring self-contained. When I'm done, io_uring_sqe is going to be IoUring.Sqe, and IoUring is exposed in linux.zig so you can access sqe with IoUring.Sqe.

bernardassan · 2025-10-11T00:42:41Z

Now have IoUring and Linux test tests passing

❯ zig test lib/std/std.zig --zig-lib-dir lib --test-filter "os.linux"
68/116 os.linux.test.test.futex2_wait...SKIP
69/116 os.linux.test.test.futex2_wake...SKIP
70/116 os.linux.test.test.futex2_requeue...SKIP
111/116 os.linux.IoUring.test.waitid...SKIP
116/116 os.linux.IoUring.test.bind/listen/connect...SKIP
111 passed; 5 skipped; 0 failed.

Full test suite also passes

❯ zig test lib/std/std.zig --zig-lib-dir lib
441/2932 debug.test.manage resources correctly...SKIP
1163/2932 crypto.25519.x25519.test.rfc7748 1,000 iterations...SKIP
1164/2932 crypto.25519.x25519.test.rfc7748 1,000,000 iterations...SKIP
1288/2932 crypto.scrypt.test.kdf...SKIP
1289/2932 crypto.scrypt.test.kdf rfc 1...SKIP
1290/2932 crypto.scrypt.test.kdf rfc 2...SKIP
1291/2932 crypto.scrypt.test.kdf rfc 3...SKIP
1292/2932 crypto.scrypt.test.kdf rfc 4...SKIP
1293/2932 crypto.scrypt.test.password hashing (crypt format)...SKIP
1294/2932 crypto.scrypt.test.strHash and strVerify...SKIP
1295/2932 crypto.scrypt.test.unix-scrypt...SKIP
1301/2932 crypto.pbkdf2.test.RFC 6070 16,777,216 iterations...SKIP
1304/2932 crypto.pbkdf2.test.Very large dk_len...SKIP
1408/2932 fs.test.test.delete a setAsCwd directory on Windows...SKIP
1409/2932 fs.test.test.invalid UTF-8/WTF-8 paths...SKIP
1832/2932 net.test.test.non-blocking tcp server...SKIP
1833/2932 posix.test.test.WTF-8 to WTF-16 conversion buffer overflows...SKIP
1837/2932 posix.test.test.readlink on Windows...SKIP
1861/2932 posix.test.test.POSIX file locking with fcntl...SKIP
1941/2932 os.linux.test.test.futex2_wait...SKIP
1942/2932 os.linux.test.test.futex2_wake...SKIP
1943/2932 os.linux.test.test.futex2_requeue...SKIP
1945/2932 os.windows.nls.test.upcaseW matches RtlUpcaseUnicodeChar...SKIP
2269/2932 json.static_test.test.test all types...SKIP
2305/2932 json.dynamic_test.test.polymorphic parsing...SKIP
2363/2932 os.linux.IoUring.test.waitid...SKIP
2368/2932 os.linux.IoUring.test.bind/listen/connect...SKIP
2905 passed; 27 skipped; 0 failed.
4 fuzz tests found.

RaidoAun · 2025-10-17T10:38:44Z

There currently does not seem to be a replacement for int_flags on the ring. liburing uses that to set NO_IOWAIT flag to then before doing enter in for example submit_and_wait instead of initializing flags as 0 it is instead set to a mask of int_flags which includes NO_IOWAIT.
see:
#25604

and:
axboe/liburing@8f0c124#diff-ddb623c7d8d649eff38f34b19b7a2f8b55e1d448ee94b813a9541d5304467629

otherwise I like these changes since they add type safety around the flags. While playing with this I found the bug in my own PR.

bernardassan · 2025-10-17T14:21:24Z

This is on my to-do list, but I wanted to first get IoUring up-to-date with liburing 2.12 before working on these changes. Because I realized some liburing operations (Check commit message 6bb7f30) depend on int_flags and enter_ring_fd.

But because this is affecting a real-world projects ghostty which I use 😄 , let me add the set_io_await functionality that you PR here #25604 quickly so you can test it out.

bernardassan · 2025-10-17T16:43:33Z

@RaidoAun a port of your solution in #25604 is ready in this branch

And can you share your GitHub email so I add you as a Co-Author of that commit

RaidoAun · 2025-10-17T18:36:10Z

Thanks, I will take a look.

Im fine without being co authored, but if you insist then you can just use my public github email ([email protected]). It should also be visible on the commit on my PR.

lib/std/os/linux/IoUring.zig

RaidoAun

The code regarding NO_IOWAIT looks fine now. If I find time to experiment with io_uring then I will try to test more of this branch. This branch has gotten quite big and I think you should try to get a core team members opinions on this. I would really like to see these changes get merged.
If you need help with this then I am willing to also contribute.

bernardassan · 2025-10-18T14:53:32Z

That is true, I didn't expect it to get this big, but I guess because we are way behind liburing 2.12 that should have been expected. @RaidoAun, a helping hand is always welcome. It would be nice to have an extra hand on deck when working on the future enhancements stated in the PR description.

I agree it's a good time to start getting reviews from core team members and previous contributors, as there are only a few more operations I want to implement and leave the rest as future enhancements, which I would make another PR for.

bernardassan · 2025-10-18T15:12:25Z

cc @mlugg @ianic @andrewrk @nikneym @topecongiro @ozgrakkurt @psnszsn @joadnacer @Vexu @jacobly0 @mllken @vrischmann @hnakamur @lithdew @ayende @jorangreef @ifreund @daurnimator

bernardassan · 2025-10-18T15:16:57Z

On my Arch Linux machine

❯ zig test lib/std/std.zig --zig-lib-dir lib --test-filter "os.linux"
All 117 tests passed.

lib/std/os/linux/IoUring.zig

IoUring Overhaul Named Constants for IoUring Move io_uring_sqe.zig into IoUring Add new exposed IoUring data types Improve naming and namespacing a bit Improve api of init with Setup.Flags // axboe/liburing#1075 (comment) // axboe/liburing#811 (comment) // https://nick-black.com/dankwiki/index.php/Io_uring Catch simple incompatible flag configurations during init_params Start updating IoUring gradually to use the new typed flags Slowly remove IoUring bits and pieces from linux.zig Signed-off-by: Bernard Assan <[email protected]>

and use linux Flags where appropriate constants is a namespace make AsyncCancel flags more descriptive move cqe.flags into the Cqe type make io_uring_sqe ioprio a packed union of SendRecv & Accept TODO: move some of the used linux Flags to be packed structs Signed-off-by: Bernard Assan <[email protected]>

Improve naming and namespacing of IoUring flags and types to match Zig Style move some flags from uflags to its type Add matches doc comments to make it easy to match Zig IoUring types to liburing Signed-off-by: Bernard Assan <[email protected]>

Add RenameFlags for IoUring renameat Start update io_uring Types in linux.zig to use IoUring Continue change options and flags to Typed types Fix test cases to use updated IoUring flags and options Signed-off-by: Bernard Assan <[email protected]>

Update std to use Statx.Mask Update statx syscall to use Statx.Mask Signed-off-by: Bernard Assan <[email protected]>

Use the new Flag types in IoUring fix some test Signed-off-by: Bernard Assan <[email protected]>

Add Shut, PF, AF, IPPROTO types Update IoUring and test to use some of this flags TODO: avoid breaking API and make transitioning smooth Instead mark the previous API as deprecated for at least One zig release while using the new in the Zig codebase Signed-off-by: Bernard Assan <[email protected]>

This allows for a gradual transitioning to the new typed flags Signed-off-by: Bernard Assan <[email protected]>

Signed-off-by: Bernard Assan <[email protected]>

4❯ zig test lib/std/std.zig --zig-lib-dir lib --test-filter "os.linux" 68/116 os.linux.test.test.futex2_wait...SKIP 69/116 os.linux.test.test.futex2_wake...SKIP 70/116 os.linux.test.test.futex2_requeue...SKIP 111/116 os.linux.IoUring.test.waitid...SKIP 116/116 os.linux.IoUring.test.bind/listen/connect...SKIP 111 passed; 5 skipped; 0 failed. fix skipKernelLessThan fn to work on wsl linux fix logical error in flags compatibility check Update all IoUring to use the implemented flags bandage types that affect posix for now so that the scope doesn't become unmanageable TODO: impl linux.SOL and linux.SO types Signed-off-by: Bernard Assan <[email protected]>

Replace SO with So type and Add SO named constants for compatability Replace SOL with Sol type and Add SOL named constants for compatability Replace some more typed flags with their typed equivalent Remove any unnecessary IoUring dependency on linux.* Add TODOs for all untyped flags Signed-off-by: Bernard Assan <[email protected]>

Prefer Using aliases over packed union field Fix bug in HANDLE_FID declaration Use EpollOp op type and Epoll packed struct flag type in IoUring Signed-off-by: Bernard Assan <[email protected]>

Add Deprecated Op constants for compatibility Signed-off-by: Bernard Assan <[email protected]>

Signed-off-by: Bernard Assan <[email protected]>

update io_uring_enter and io_uring_register syscalls to use IoUring flags and types Cleanup comments Signed-off-by: Bernard Assan <[email protected]>

Signed-off-by: Bernard Assan <[email protected]>

now that IoUring.zig is self contained Signed-off-by: Bernard Assan <[email protected]>

add Deprecated linux.W fns Signed-off-by: Bernard Assan <[email protected]>

Signed-off-by: Bernard Assan <[email protected]>

update `cq_ring_needs_flush()` to match liburing replace FilesUpdate with RsrcUpdate struct Signed-off-by: Bernard Assan <[email protected]>

Future enhancement: enter_ring_fd: u24, init_flags: u8, register functions that require the extra fields above like io_uring_register_ring_fd, io_uring_close_ring_fd, io_uring_clone_buffers_offset, io_uring_set_iowait not implemented register_*_tag functions not implemented remove unnecessary use of @ptrCast in io_uring_register Signed-off-by: Bernard Assan <[email protected]>

Signed-off-by: Bernard Assan <[email protected]>

add IoUring tee syscall functionality adapt ReadBuffer union idea for read_fixed to support fixed read and readv Use same idea and create a WriteBuffer type for write() and write_fixed() ignored flags for splice and tee lets see if they become important in the future add a get_data helper to Cqe Signed-off-by: Bernard Assan <[email protected]>

Add timeout update, cancel_fd, epoll_wait, files_update, open, open_direct, f/madvice don't implement f/madvice64 for now I doubt it is used by a lot of people in practice Implement BufferGroup read_multishot Try to make definition of function align as logically as posible with liburing to make keeping our implementation in sync easy TODO: add M/Fadvice enum flags TODO: understand buffer_selection and how its different from BufferGroup Signed-off-by: Bernard Assan <[email protected]>

ignore prep_openat2* for now add non-at variant of some unlink, rename, mkdir, symlink, link TODO: add Sync File Flags Signed-off-by: Bernard Assan <[email protected]>

TODO: add SetXattr flags Signed-off-by: Bernard Assan <[email protected]>

closes ziglang#25566 closes ziglang#25604 and IoUring enter_flags() helper function add typed Init flags which match int_flags in liburing Co-authored-by: RaidoAun <[email protected]> Signed-off-by: Bernard Assan <[email protected]>

Signed-off-by: Bernard Assan <[email protected]>

Signed-off-by: Bernard Assan <[email protected]> Add futex_* operations to IoUring Refactor Futex 2 flags into Futex2 struct add mpol to Wait flags and fix private field as its 128 not 32 Signed-off-by: Bernard Assan <[email protected]> Update futex2_* functions and constants to use the new Futex2 type Improve the Api of futex2_* functions to be more idiomatic Zig Signed-off-by: Bernard Assan <[email protected]>

Cleanup linux/test.zig type futex/2_* uaddr as *const u32 consider changing to *const atomic.Value(u32) Use At Flags in fstatat Use EpollOp in epoll_ctl syscall Signed-off-by: Bernard Assan <[email protected]>

Build on the extensive work already done IoUring: add fixed_fd_install, ftruncate, cmd_discard Working on IoUring pipe flags Signed-off-by: Bernard Assan <[email protected]>

Add dedicated Pipe2 flags for pipe2 as O contains some fields which are invalid in that context Signed-off-by: Bernard Assan <[email protected]>

Signed-off-by: Bernard Assan <[email protected]>

Use explicit array types were possible Signed-off-by: Bernard Assan <[email protected]>

bernardassan force-pushed the iouring-overhaul branch from 956b5a1 to dbd89a6 Compare October 11, 2025 17:24

bernardassan force-pushed the iouring-overhaul branch from c43fdb4 to 0768827 Compare October 17, 2025 16:29

bernardassan force-pushed the iouring-overhaul branch from 0768827 to 53b2359 Compare October 17, 2025 16:46

bernardassan force-pushed the iouring-overhaul branch from 53b2359 to 5785357 Compare October 17, 2025 19:10

RaidoAun suggested changes Oct 17, 2025

View reviewed changes

lib/std/os/linux/IoUring.zig Outdated Show resolved Hide resolved

lib/std/os/linux/IoUring.zig Outdated Show resolved Hide resolved

lib/std/os/linux/IoUring.zig Outdated Show resolved Hide resolved

lib/std/os/linux/IoUring.zig Outdated Show resolved Hide resolved

bernardassan force-pushed the iouring-overhaul branch from 5785357 to dba0c81 Compare October 17, 2025 23:14

bernardassan requested a review from RaidoAun October 17, 2025 23:17

RaidoAun approved these changes Oct 18, 2025

View reviewed changes

bernardassan marked this pull request as ready for review October 18, 2025 14:55

bernardassan changed the title ~~IoUring: use typed Flags and Features for IoUring~~ IoUring: use typed Flags and start syncing with liburing 2.12 Oct 18, 2025

RaidoAun reviewed Oct 19, 2025

View reviewed changes

lib/std/os/linux/IoUring.zig Outdated Show resolved Hide resolved

bernardassan added 9 commits October 21, 2025 11:07

Improve organization of fn and structs in IoUring

aad025a

Update std to use Statx.Mask Update statx syscall to use Statx.Mask Signed-off-by: Bernard Assan <[email protected]>

Replace AT,W,SHUT,SOCK with a packed struct Flag type

7d98523

Use the new Flag types in IoUring fix some test Signed-off-by: Bernard Assan <[email protected]>

Restore deprecated contants using new Flag types

b955207

This allows for a gradual transitioning to the new typed flags Signed-off-by: Bernard Assan <[email protected]>

Replace MSG with Packed Struct Flags

a9b1833

Signed-off-by: Bernard Assan <[email protected]>

bernardassan and others added 22 commits October 21, 2025 11:07

Replace EPOLL struct with an EpollOp enum and Epoll packed struct type

7b25d92

Prefer Using aliases over packed union field Fix bug in HANDLE_FID declaration Use EpollOp op type and Epoll packed struct flag type in IoUring Signed-off-by: Bernard Assan <[email protected]>

fix error of setting some fields in Epoll to true by default

e0fa257

Add Deprecated Op constants for compatibility Signed-off-by: Bernard Assan <[email protected]>

Remove io_uring_sqe.zig from CMakeLists

b2776b0

Signed-off-by: Bernard Assan <[email protected]>

Use lower case identifiers for IoUring flags and enums

86638f0

update io_uring_enter and io_uring_register syscalls to use IoUring flags and types Cleanup comments Signed-off-by: Bernard Assan <[email protected]>

replace direct set of some flags with link_next and set_flags calls

b869a6d

Signed-off-by: Bernard Assan <[email protected]>

Remove io_uring bit and pieces from linux.zig

bf49873

now that IoUring.zig is self contained Signed-off-by: Bernard Assan <[email protected]>

Fix posix.W in process/Child.zig

7b588a8

add Deprecated linux.W fns Signed-off-by: Bernard Assan <[email protected]>

Add mips defination for Epoll

e650ea2

Signed-off-by: Bernard Assan <[email protected]>

Add and improve comments

e59fa49

Signed-off-by: Bernard Assan <[email protected]>

Remove unnecessary use of @as coercion

da1a3f3

update `cq_ring_needs_flush()` to match liburing replace FilesUpdate with RsrcUpdate struct Signed-off-by: Bernard Assan <[email protected]>

Remove unnecessary null to optional anyopaque coercion

fa46822

Signed-off-by: Bernard Assan <[email protected]>

add IoUring send_bundle, send_to, recv_multishot, sync_file_range

e85d5fa

ignore prep_openat2* for now add non-at variant of some unlink, rename, mkdir, symlink, link TODO: add Sync File Flags Signed-off-by: Bernard Assan <[email protected]>

add msg_ring_*, setxattr and getxattr IoUring operations

00e948c

TODO: add SetXattr flags Signed-off-by: Bernard Assan <[email protected]>

IoUring: Implement set_iowait functionality

caf492e

closes ziglang#25566 closes ziglang#25604 and IoUring enter_flags() helper function add typed Init flags which match int_flags in liburing Co-authored-by: RaidoAun <[email protected]> Signed-off-by: Bernard Assan <[email protected]>

Add XattrSource to decide how to prepare set/getxattr operations

6a4cee3

Signed-off-by: Bernard Assan <[email protected]>

Add a Futex2 Bitset type for futex2 wake and wait syscalls

321a739

Cleanup linux/test.zig type futex/2_* uaddr as *const u32 consider changing to *const atomic.Value(u32) Use At Flags in fstatat Use EpollOp in epoll_ctl syscall Signed-off-by: Bernard Assan <[email protected]>

bernardassan force-pushed the iouring-overhaul branch from e0f8ccb to 321a739 Compare October 21, 2025 18:13

bernardassan mentioned this pull request Oct 23, 2025

Linux: Nuke Stat bits in favour of Statx #25639

Open

bernardassan added 4 commits October 23, 2025 17:31

IoUring: Working on Pipe2 flags

18c2d02

Build on the extensive work already done IoUring: add fixed_fd_install, ftruncate, cmd_discard Working on IoUring pipe flags Signed-off-by: Bernard Assan <[email protected]>

IoUring: implement pipe and pipe_direct operations

ad7c24e

Add dedicated Pipe2 flags for pipe2 as O contains some fields which are invalid in that context Signed-off-by: Bernard Assan <[email protected]>

Merge archs that have the same Pipe2 flags

ababd46

Signed-off-by: Bernard Assan <[email protected]>

Use linux types directly since IoUring is only supported on linux

e71a82d

Use explicit array types were possible Signed-off-by: Bernard Assan <[email protected]>

Uh oh!

IoUring: use typed Flags and start syncing with liburing 2.12 #25394

Are you sure you want to change the base?

IoUring: use typed Flags and start syncing with liburing 2.12 #25394

Conversation

bernardassan commented Sep 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

IoUring Overhaul

Future Enhancement

Uh oh!

daurnimator commented Sep 29, 2025

Uh oh!

bernardassan commented Sep 29, 2025

Uh oh!

bernardassan commented Oct 11, 2025

Uh oh!

RaidoAun commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bernardassan commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bernardassan commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RaidoAun commented Oct 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RaidoAun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bernardassan commented Oct 18, 2025

Uh oh!

bernardassan commented Oct 18, 2025

Uh oh!

bernardassan commented Oct 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bernardassan commented Sep 28, 2025 •

edited

Loading

RaidoAun commented Oct 17, 2025 •

edited

Loading

bernardassan commented Oct 17, 2025 •

edited

Loading

bernardassan commented Oct 17, 2025 •

edited

Loading

RaidoAun left a comment •

edited

Loading